Papers with model ensemble
Leveraging Large Language Models for Conversational Multi-Doc Question Answering: The First Place of WSDM Cup 2024 (2025.findings-acl)
Copied to clipboard
| Challenge: | WSDM Cup 2024 presents a challenge for conversational multi-doc question answering using large language models . a hybrid training strategy is developed to make the most of in-domain unlabeled data . |
| Approach: | They propose a conversational multi-doc question answering challenge in WSDM Cup 2024 . they adapt LLMs to the task, then devise a hybrid training strategy to make the most of unlabeled data. |
| Outcome: | The proposed approach ranked 1st in the WSDM Cup 2024 challenge . it exploits the superior natural language understanding and generation capability of Large Language Models . |
Are Pre-trained Language Models Useful for Model Ensemble in Chinese Grammatical Error Correction? (2023.acl-short)
Copied to clipboard
| Challenge: | Model ensemble has been in widespread use for Grammatical Error Correction (GEC), boosting model performance. |
| Approach: | They propose to use model ensembles computed by pre-trained language models to improve model performance. |
| Outcome: | The proposed ensembles do not improve but get worse after the PLM-based ensemble. |
Single Model Ensemble for Subword Regularized Models in Low-Resource Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing subword regularizations use multiple segmentations during training but only use one segmentation in inference. |
| Approach: | They propose an inference strategy that uses multiple subword segmentations to solve this discrepancy in the training process and inference. |
| Outcome: | The proposed strategy reduces the cost of training and improves the performance of models trained with subword regularization in low-resource machine translation tasks. |
Single Model Ensemble using Pseudo-Tags and Distinct Vectors (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for model ensembles require time, memory, and management effort to perform tasks. |
| Approach: | They propose a method that replicates the effects of a model ensemble with a single model. |
| Outcome: | The proposed method emulates or outperforms a traditional model ensemble with 1/K-times fewer parameters on text classification and sequence labeling tasks. |
Decoding Reading Goals from Eye Movements (2025.acl-long)
Copied to clipboard
| Challenge: | a study examines whether readers can distinguish between two types of reading goals: information seeking and ordinary reading for comprehension. |
| Approach: | They propose a method to distinguish between two types of reading goals: information seeking and ordinary reading for comprehension. |
| Outcome: | The proposed model solves the reading goal-oriented task with the most accurate predictions in real time, the authors say . |
Delta-training: Simple Semi-Supervised Text Classification using Pretrained Word Embeddings (D19-1)
Copied to clipboard
| Challenge: | Pretrained word embeddings outperforms classifiers with randomly initialized word embeds, a new method is proposed for semi-supervised text classification. |
| Approach: | They propose a method that uses pretrained word embeddings to predict text classification . they use unlabeled data to build a classifier, and use early-stopping to improve performance . |
| Outcome: | The proposed method outperforms self-training and co-training frameworks on unlabeled data. |
CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing (2022.acl-long)
Copied to clipboard
| Challenge: | Existing work has resorted to sharing weights among models, but results are not affordable for real-world deployment. |
| Approach: | They propose a consistency-regularized ensemble learning approach based on perturbed models to retain ensemble benefits while maintaining a low memory cost. |
| Outcome: | The proposed approach outperforms the standard ensemble of 8 BERT-base models on the GLUE benchmark by 0.7 with a significantly smaller model size. |
Regression Bugs Are In Your Model! Measuring, Reducing and Analyzing Regressions In NLP Model Updates (2021.acl-long)
Copied to clipboard
| Challenge: | Using negative flips, we quantify, reduce and analyze regression errors in deep neural networks. |
| Approach: | They propose to quantify, reduce and analyze regression errors in NLP models by negative flips. |
| Outcome: | The proposed model update regression has a prevalent presence across tasks in the GLUE benchmark. |
One for All: Update Parameterized Knowledge Across Multiple Models with Once Edit (2025.acl-long)
Copied to clipboard
Weitao Ma, Xiyuan Du, Xiaocheng Feng, Lei Huang, Yichong Huang, Huiyi Zhang, Xiaoliang Yang, Baohang Li, Xiachong Feng, Ting Liu, Bing Qin
| Challenge: | Existing methods for modifying large language models focus on individual models, resulting in errors and hallucinations. |
| Approach: | They propose an ensemble-based approach that employs a plug-in model as the editing module and a dynamic weight mechanism to enhance its effectiveness. |
| Outcome: | The proposed approach outperforms existing methods while achieving superior editing efficiency. |
LM-Combiner: A Contextual Rewriting Model for Chinese Grammatical Error Correction (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent work using model ensemble methods based on voting can effectively mitigate over-correction and improve the precision of the GEC system. |
| Approach: | They propose a rewriting model that can directly modify the over-correction of GEC system outputs without a model ensemble. |
| Outcome: | The proposed model can mitigate over-correction and improve accuracy of Chinese grammatical error correction tasks without a model ensemble. |
Recurrent Knowledge Identification and Fusion for Language Model Continual Learning (2025.acl-long)
Copied to clipboard
Yujie Feng, Xujia Wang, Zexin Lu, Shenghong Fu, Guangyuan Shi, Yongxin Xu, Yasha Wang, Philip S. Yu, Xu Chu, Xiao-Ming Wu
| Challenge: | Continual learning (CL) is crucial for large language models without costly retraining. |
| Approach: | They propose a framework for recurrent knowledge identification and fusion that enables dynamic estimation of parameter importance distributions to enhance knowledge transfer. |
| Outcome: | The proposed framework mitigates catastrophic forgetting and enhances knowledge transfer. |